Skip to content

Migrate to spago v1#316

Open
pete-murphy wants to merge 7 commits into
purescript:masterfrom
pete-murphy:migrate-to-spago-v1
Open

Migrate to spago v1#316
pete-murphy wants to merge 7 commits into
purescript:masterfrom
pete-murphy:migrate-to-spago-v1

Conversation

@pete-murphy

@pete-murphy pete-murphy commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Description of the change

Resolves #315

This migrates the client and staging Spago projects to use latest (1.0.4), and updates compiler version and package set to latest while I was at it. Note that 0.15.15 is latest version available on Hackage at time of writing.

For client updates, I just followed these instructions in Spago repo: https://github.com/purescript/spago#migrate-from-spagodhall-to-spagoyaml.

For staging/Haskell updates I followed the instructions in RELEASE.md that I also updated in these changes. I did not change stack.yaml resolver because 0.15.15 is still on lts-20.9.

This is a large diff, but most of that is in generated files. Ignoring staging/* (effectively generated from following the RELEASE.md instructions) and client/spago.lock, the diff is much more reasonable:

$ git diff origin/master --stat ':(exclude)staging/*' ':(exclude)client/spago.lock'

 .github/workflows/ci.yml              |  4 +--
 CHANGELOG.md                          |  3 ++
 README.md                             |  9 +++---
 RELEASE.md                            | 72 ++++++++++++++++++++++++--------------------
 client/config/dev/Try.Config.purs     | 15 ---------
 client/config/prod/Try.Config.purs    | 15 ---------
 client/package.json                   | 19 ++++++------
 client/packages.dhall                 |  5 ---
 client/spago.dhall                    | 45 ---------------------------
 client/spago.yaml                     | 48 +++++++++++++++++++++++++++++
 client/src/Try/Config.js              | 19 ++++++++++++
 client/src/Try/Config.purs            |  5 +++
 client/src/Try/SharedConfig.purs      |  7 +++--
 client/updateSharedConfigVersions.mjs | 20 +++++++++---
 stack.yaml                            |  2 +-
 stack.yaml.lock                       |  8 ++---
 16 files changed, 154 insertions(+), 142 deletions(-)

That said, I am open splitting this up into separate MRs.


Checklist:

  • Added the change to the changelog's "Unreleased" section with a reference to this PR (e.g. "- Made a change (#0 by @)")
  • Linked any existing issues or proposals that this pull request should close
  • Updated or added relevant documentation
  • Added a test for the contribution (if applicable)

@pete-murphy pete-murphy force-pushed the migrate-to-spago-v1 branch 3 times, most recently from 46dbaec to 7f9b9a5 Compare June 6, 2026 13:11
Comment thread README.md Outdated
Comment thread client/src/Try/Config.js
@pete-murphy pete-murphy force-pushed the migrate-to-spago-v1 branch from 7f9b9a5 to 11e2e80 Compare June 13, 2026 19:39
@pete-murphy pete-murphy force-pushed the migrate-to-spago-v1 branch from 429d4ab to 9747a47 Compare June 13, 2026 20:25
@pete-murphy

pete-murphy commented Jun 13, 2026

Copy link
Copy Markdown
Contributor Author

Oh no, I tried running locally with same memory settings as production build (set -o noglob && stack exec trypurescript -- +RTS -N2 -A128m -M3G -s -RTS 8081 $(spago sources)) and ran out of memory.

❯ (set -o noglob && stack exec trypurescript -- +RTS -N2 -A128m -M3G -s -RTS 8081 $(spago sources))
Reading Spago workspace configuration...

✓ Selecting package to build: try-purescript-server


Warning: nix (Nix) 2.34.7+1 is on the PATH (at /run/current-system/sw/bin/nix)
         but Stack's Nix integration is disabled. To mute this message in
         future, set notify-if-nix-on-path: false in Stack's configuration.
         
trypurescript: Heap exhausted;
trypurescript: Current maximum heap size is 3221225472 bytes (3072 MB).
trypurescript: Use `+RTS -M<size>' to increase it.
  36,907,900,144 bytes allocated in the heap
   7,635,067,328 bytes copied during GC
   3,479,980,704 bytes maximum residency (28 sample(s))
      21,874,656 bytes maximum slop
            3739 MiB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0       108 colls,    42 par    2.344s   3.292s     0.0305s    0.1313s
  Gen  1        28 colls,     5 par   39.524s  111.686s     3.9888s    7.0771s

  Parallel GC work balance: 86.50% (serial 0%, perfect 100%)

  TASKS: 9 (1 bound, 8 peak workers (8 total), using -N2)

  SPARKS: 7427 (7381 converted, 0 overflowed, 0 dud, 0 GC'd, 46 fizzled)

  INIT    time    0.001s  (  0.080s elapsed)
  MUT     time   42.664s  (  9.678s elapsed)
  GC      time   41.868s  (114.978s elapsed)
  EXIT    time    0.002s  (  0.005s elapsed)
  Total   time   84.534s  (124.742s elapsed)

  Alloc rate    865,091,047 bytes per MUT second

  Productivity  50.5% of total user, 7.8% of total elapsed

Running with -M4G succeeds.

@pete-murphy pete-murphy marked this pull request as ready for review June 15, 2026 22:37
@pete-murphy

pete-murphy commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Dumping the result of investigating this with Opus: https://gist.github.com/pete-murphy/bd776708d72a00a80ba1b7e7845f2982. Option 3 seems the most attractive if it could work.

@thomashoneyman

Copy link
Copy Markdown
Member

Option 3 is indeed attractive and I’m perfectly happy experimenting with reworking try PureScript, it’s definitely not received a lot of love recently.

I wouldn’t dismiss option 2 offhand — Try PureScript is browser based so certainly node packages, etc. can safely be excluded

@pete-murphy

pete-murphy commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

I wouldn’t dismiss option 2 offhand — Try PureScript is browser based so certainly node packages, etc. can safely be excluded

I did end up looking into Option 2 after all since it seems much easier and I am less likely to get to the harder Option 3 in near term. If I trim out Node packages & some large packages that have no dependents, I am able to run the server with (set -o noglob && stack exec trypurescript -- +RTS -N2 -A128m -M3G -s -RTS 8081 $(spago sources)) without exceeding that max heap size. The result was this commit: 0fd79d9.

The non-Node packages (largest packages with no dependents) that were omitted:

  • framer-motion
  • motion
  • next-purs-rsc
  • react-basic-dom-beta
  • react-icons
  • css-frameworks
  • lumi-components
  • yoga-sqlite
  • rito

#4269d0 are packages that were kept
#e0820e is omitted (because Node)
#c33d3d is omitted (because big package with no dependents)

image

Here's a notebook for exploring what packages to omit if it's useful: https://new.observablehq.com/@pete-murphy/try-pure-script-package-set-trim-explorer (mostly made by Opus 4.8)

The "no dependents" criterion is arbitrary, open to ideas if you can think of something that would be more fair? (Age of package, some other popularity metric...)

@thomashoneyman

thomashoneyman commented Jul 3, 2026

Copy link
Copy Markdown
Member

Quick followup on your questions in the gist:

  1. The droplet has 3.8 GiB RAM (and no swap, though we could add it), so bumping to -M5G is off the table for now; we already overpay for trypurescript IMO so I don't want to resize upward unless absolutely necessary. So I think we should do the trimmed package set path.

  2. I looked, and production is actually already occasionally being OOM-killed with the current package set. journalctl shows the service dying with status=9/KILL four times on April 28 and again on May 24, with Restart=always quietly recovering each time. So the existing -M3G cap plus other overhead is already (occasionally) insufficient!

  3. I also checked the staging/output directory, and this doesn't exist on the box

I think the option 2 is the way to go, but before we merge, could you measure the trimmed set's peak (maxLive from +RTS -s) on a cold boot (no output)? If it's much above ~2.5 GB, it's probably worth trimming a little further.

Longer term, option 3 looks even better, but we don't need to do it now.

@thomashoneyman

thomashoneyman commented Jul 3, 2026

Copy link
Copy Markdown
Member

Got a little access to Fable, so ran it over this change — here are results, lightly proofread:

<---- fable ---->
Following up on the trim-criteria question with some data. I scanned every package in the untrimmed set (638 packages) for bare JS import specifiers in their foreign modules — i.e. npm/Node imports that Try PureScript cannot resolve at runtime, since the playground ships no node_modules. One important nuance: client/public/frame.html already has a JSPM import map that shims exactly seven specifiers (react, react-dom, react-dom/client, react-dom/server, big-integer, decimal.js, uuid), so the disqualifying condition is a bare import not covered by the import map.

Results, combined with the dependency graph from spago.lock:

  • 118 packages directly contain unshimmed bare imports. Including their transitive dependents, the "cannot run in the playground" closure is 148 packages / 3,658 modules — nearly half the full set's 7,426 modules.
  • The current trim in 0fd79d9 cuts 79 packages / 2,945 modules, but 75 of the packages it keeps are in the cannot-run closure — they typecheck here but throw the moment you run them.
  • Conversely, only 6 cut packages are actually runnable: spec, spec-mocha, test-unit, pmock, react-basic-dom-beta (all its imports are shimmed), and css-frameworks (runs, though its class names do nothing without the frameworks' stylesheets in the frame).

So my suggestion: trim by "cannot execute in the playground" instead of size/dependents. It cuts ~24% more modules than the current trim (more memory headroom), it's fully principled — nothing is removed for being big or unpopular, only for being unable to run — and it lets spec, test-unit, and react-basic-dom-beta come back.

Two caveats:

  1. Package-level is an upper bound. Some flagged packages have a single broken module (e.g. which touching child_process); a user who never imports that module is unaffected. Fine to keep borderline popular ones.
  2. "Cannot run" is fixable, not a death sentence. Browser-friendly packages (chartjs, xterm, idb, nanoid, markdown-it-js, react-markdown, …) could be rescued with one import-map entry each via the JSPM generator. That also gives package authors a concrete path to inclusion, which answers the fairness question better than any popularity metric.
75 packages kept by the current trim that cannot run
Package Unshimmed imports (or broken dependency)
ace ace-builds, ace/anchor, ace/background_tokenizer, ace/document, ace/edit_session
address-rfc2821 address-rfc2821
chartjs chart.js/auto
chartjs-halogen depends on chartjs
choku chalk
debounce debouncing
deno https://deno.land/std@0.144.0/crypto/mod.ts, https://deno.land/std@0.144.0/dotenv/mod.ts, https://deno.land/std@0.144.0/fs/mod.ts, https://deno.land/std@0.144.0/http/server.ts, https://deno.land/std@0.145.0/datetime/mod.ts
droplet pg
echarts-simple echarts
elmish react-dom/server.js
elmish-enzyme enzyme
elmish-hooks stacktrace-parser
elmish-html depends on elmish
elmish-testing-library @happy-dom/global-registrator, react-dom/test-utils.js
elmish-time-machine depends on elmish, elmish-hooks, elmish-html
fahrtwind depends on react-basic-emotion
faker-ffi @faker-js/faker
fakerjs @faker-js/faker
gojs gojs
halogen-echarts-simple depends on echarts-simple
halogen-xterm depends on xterm
hylograph-d3-kernel d3-drag, d3-force, d3-selection
hylograph-simulation d3-drag, d3-force, d3-selection
hylograph-simulation-halogen depends on hylograph-d3-kernel, hylograph-simulation
i18next i18next, i18next-browser-languagedetector
idb idb
jsdom jsdom
leveldb level
logging-journald depends on systemd-journald
markdown-it-js markdown-it, markdown-it-collapsible
milkdown @milkdown/crepe, @milkdown/kit/core, @milkdown/kit/utils
milkis node-fetch
mysql mysql
n3 n3
nanoid nanoid
nextjs next/document, next/head, next/image.js, next/link, next/navigation
nextui @nextui-org/react, next-themes
node-sqlite3 sqlite3
oak virtual-dom/create-element, virtual-dom/diff, virtual-dom/h, virtual-dom/patch
oak-debug depends on oak
react-aria @react-aria/button, @react-aria/combobox, @react-aria/focus, @react-aria/interactions, @react-aria/listbox
react-basic-dnd react-dnd, react-dnd-html5-backend, react-dnd-test-backend, react-dnd-touch-backend
react-basic-emotion @emotion/react
react-basic-storybook @storybook/addon-actions, @storybook/addon-docs, @storybook/testing-library
react-dnd-kit @dnd-kit/abstract/modifiers, @dnd-kit/collision, @dnd-kit/dom, @dnd-kit/dom/modifiers, @dnd-kit/dom/utilities
react-markdown react-markdown, remark-breaks, remark-gfm
react-virtuoso react-virtuoso
reactix @testing-library/react, react-dom/test-utils
recharts recharts
rough-notation rough-notation
systemd-journald systemd-journald
tanstack-query @tanstack/react-query
toestand depends on reactix
ulid ulid
url-regex-safe url-regex-safe
visx @visx/annotation, @visx/axis, @visx/brush, @visx/clip-path, @visx/curve
vitest vitest
webextension-polyfill webextension-polyfill
which which
xterm xterm, xterm-addon-fit, xterm-addon-web-links, xterm-addon-webgl
yaml-next js-yaml
yoga-better-auth better-auth, better-auth/client, better-auth/db, better-auth/plugins, pg
yoga-dynamodb @aws-sdk/client-dynamodb, @aws-sdk/lib-dynamodb
yoga-elasticsearch @elastic/elasticsearch
yoga-fastify @fastify/cors, @fastify/helmet, @fastify/rate-limit, @fastify/websocket, argon2
yoga-fetch node-fetch
yoga-heroui @heroui/react
yoga-jaeger jaeger-client
yoga-next-fastify next
yoga-pino @opentelemetry/api-logs, pino
yoga-postgres pg
yoga-react-native react-native, react-native-fs, react-native-macos, twrnc
yoga-redis ioredis
yoga-shadcn cmdk, embla-carousel-react, input-otp, radix-ui, react-day-picker
z3 z3-solver
Full data: package → unshimmed bare imports, cannot-run closure, current cut list (JSON)
{
 "unshimmed": {
  "affjax-node": [
   "url",
   "xhr2"
  ],
  "yoga-pino": [
   "@opentelemetry/api-logs",
   "pino"
  ],
  "motion": [
   "motion/react",
   "motion/react-m"
  ],
  "yoga-om-strom": [
   "node:fs"
  ],
  "hylograph-d3-kernel": [
   "d3-drag",
   "d3-force",
   "d3-selection"
  ],
  "react-testing-library": [
   "@testing-library/react/pure.js",
   "@testing-library/user-event"
  ],
  "yoga-opentelemetry": [
   "@opentelemetry/api",
   "@opentelemetry/api-logs",
   "@opentelemetry/exporter-logs-otlp-http",
   "@opentelemetry/exporter-trace-otlp-http",
   "@opentelemetry/instrumentation-http",
   "@opentelemetry/instrumentation-pino",
   "@opentelemetry/resources",
   "@opentelemetry/sdk-logs",
   "@opentelemetry/sdk-node",
   "@opentelemetry/sdk-trace-base",
   "@opentelemetry/sdk-trace-node",
   "@opentelemetry/semantic-conventions"
  ],
  "node-fs": [
   "node:fs",
   "util"
  ],
  "org-doc": [
   "instaparse",
   "node:fs"
  ],
  "nanoid": [
   "nanoid"
  ],
  "react-virtuoso": [
   "react-virtuoso"
  ],
  "yoga-redis": [
   "ioredis"
  ],
  "chartjs": [
   "chart.js/auto"
  ],
  "yoga-fastify": [
   "@fastify/cors",
   "@fastify/helmet",
   "@fastify/rate-limit",
   "@fastify/websocket",
   "argon2",
   "fastify",
   "jose"
  ],
  "yoga-test-docker": [
   "child_process"
  ],
  "node-net": [
   "net",
   "node:net"
  ],
  "react-markdown": [
   "react-markdown",
   "remark-breaks",
   "remark-gfm"
  ],
  "graphql-client": [
   "@apollo/client/core/index.js",
   "@apollo/client/link/context/index.js",
   "@apollo/client/link/subscriptions/index.js",
   "@apollo/client/utilities/index.js",
   "@urql/core",
   "graphql-ws",
   "wonka"
  ],
  "yoga-fetch": [
   "node-fetch"
  ],
  "visx": [
   "@visx/annotation",
   "@visx/axis",
   "@visx/brush",
   "@visx/clip-path",
   "@visx/curve",
   "@visx/event",
   "@visx/geo",
   "@visx/gradient",
   "@visx/grid",
   "@visx/group",
   "@visx/heatmap",
   "@visx/hierarchy",
   "@visx/legend",
   "@visx/mock-data",
   "@visx/network",
   "@visx/pattern",
   "@visx/responsive",
   "@visx/scale",
   "@visx/shape",
   "@visx/stats",
   "@visx/threshold",
   "@visx/tooltip",
   "@visx/zoom",
   "d3-format",
   "d3-time-format",
   "topojson-client"
  ],
  "postgresql": [
   "buffer",
   "pg",
   "pg-copy-streams",
   "postgres-interval",
   "postgres-range",
   "stream"
  ],
  "node-child-process": [
   "node:child_process"
  ],
  "react-aria": [
   "@react-aria/button",
   "@react-aria/combobox",
   "@react-aria/focus",
   "@react-aria/interactions",
   "@react-aria/listbox",
   "@react-aria/overlays",
   "@react-aria/utils"
  ],
  "gojs": [
   "gojs"
  ],
  "echarts-simple": [
   "echarts"
  ],
  "fakerjs": [
   "@faker-js/faker"
  ],
  "yoga-heroui": [
   "@heroui/react"
  ],
  "xterm": [
   "xterm",
   "xterm-addon-fit",
   "xterm-addon-web-links",
   "xterm-addon-webgl"
  ],
  "recharts": [
   "recharts"
  ],
  "yaml-next": [
   "js-yaml"
  ],
  "react-basic-storybook": [
   "@storybook/addon-actions",
   "@storybook/addon-docs",
   "@storybook/testing-library"
  ],
  "nextui": [
   "@nextui-org/react",
   "next-themes"
  ],
  "url-regex-safe": [
   "url-regex-safe"
  ],
  "node-os": [
   "node:os",
   "node:util"
  ],
  "spec-discovery": [
   "fs",
   "path",
   "url"
  ],
  "which": [
   "which"
  ],
  "yoga-dynamodb": [
   "@aws-sdk/client-dynamodb",
   "@aws-sdk/lib-dynamodb"
  ],
  "droplet": [
   "pg"
  ],
  "oak": [
   "virtual-dom/create-element",
   "virtual-dom/diff",
   "virtual-dom/h",
   "virtual-dom/patch"
  ],
  "yoga-jaeger": [
   "jaeger-client"
  ],
  "simple-ulid": [
   "crypto"
  ],
  "yoga-postgres": [
   "pg"
  ],
  "node-zlib": [
   "node:zlib"
  ],
  "elmish-hooks": [
   "stacktrace-parser"
  ],
  "debounce": [
   "debouncing"
  ],
  "yoga-react-native": [
   "react-native",
   "react-native-fs",
   "react-native-macos",
   "twrnc"
  ],
  "rough-notation": [
   "rough-notation"
  ],
  "reactix": [
   "@testing-library/react",
   "react-dom/test-utils"
  ],
  "framer-motion": [
   "motion/react"
  ],
  "choku": [
   "chalk"
  ],
  "hylograph-simulation": [
   "d3-drag",
   "d3-force",
   "d3-selection"
  ],
  "yoga-next-fastify": [
   "next"
  ],
  "yoga-shadcn": [
   "cmdk",
   "embla-carousel-react",
   "input-otp",
   "radix-ui",
   "react-day-picker",
   "react-resizable-panels",
   "sonner",
   "vaul"
  ],
  "node-readline": [
   "node:readline"
  ],
  "node-human-signals": [
   "os"
  ],
  "next-purs-rsc": [
   "next/cache",
   "next/dynamic",
   "next/font/google",
   "next/font/local",
   "next/headers",
   "next/image",
   "next/link",
   "next/navigation",
   "next/og",
   "next/script",
   "next/server",
   "next/web-vitals"
  ],
  "systemd-journald": [
   "systemd-journald"
  ],
  "node-sqlite3": [
   "sqlite3"
  ],
  "ink": [
   "ink"
  ],
  "address-rfc2821": [
   "address-rfc2821"
  ],
  "node-http": [
   "node:http",
   "node:https"
  ],
  "deno": [
   "https://deno.land/std@0.144.0/crypto/mod.ts",
   "https://deno.land/std@0.144.0/dotenv/mod.ts",
   "https://deno.land/std@0.144.0/fs/mod.ts",
   "https://deno.land/std@0.144.0/http/server.ts",
   "https://deno.land/std@0.145.0/datetime/mod.ts",
   "https://deno.land/std@0.145.0/log/mod.ts",
   "https://deno.land/std@0.145.0/uuid/mod.ts"
  ],
  "elmish-enzyme": [
   "enzyme"
  ],
  "yoga-config": [
   "smol-toml",
   "yaml"
  ],
  "cbor-stream": [
   "cbor-x"
  ],
  "blessed": [
   "@shamansir/everblessed",
   "blessed",
   "fs",
   "path",
   "reblessed",
   "terminal-size",
   "url",
   "util"
  ],
  "node-tls": [
   "node:tls"
  ],
  "elmish-testing-library": [
   "@happy-dom/global-registrator",
   "react-dom/test-utils.js"
  ],
  "node-process": [
   "process"
  ],
  "node-buffer": [
   "node:buffer",
   "util"
  ],
  "react-dnd-kit": [
   "@dnd-kit/abstract/modifiers",
   "@dnd-kit/collision",
   "@dnd-kit/dom",
   "@dnd-kit/dom/modifiers",
   "@dnd-kit/dom/utilities",
   "@dnd-kit/geometry",
   "@dnd-kit/helpers",
   "@dnd-kit/react",
   "@dnd-kit/react/sortable"
  ],
  "rito": [
   "three/examples/jsm/loaders/GLTFLoader.js",
   "three/examples/jsm/postprocessing/BloomPass.js",
   "three/examples/jsm/postprocessing/EffectComposer.js",
   "three/examples/jsm/postprocessing/GlitchPass.js",
   "three/examples/jsm/postprocessing/RenderPass.js",
   "three/examples/jsm/postprocessing/ShaderPass.js",
   "three/examples/jsm/postprocessing/UnrealBloomPass.js",
   "three/examples/jsm/renderers/CSS2DRenderer.js",
   "three/examples/jsm/renderers/CSS3DRenderer.js",
   "three/src/cameras/PerspectiveCamera.js",
   "three/src/core/BufferAttribute.js",
   "three/src/core/BufferGeometry.js",
   "three/src/core/InstancedBufferAttribute.js",
   "three/src/core/Raycaster.js",
   "three/src/geometries/BoxGeometry.js",
   "three/src/geometries/CapsuleGeometry.js",
   "three/src/geometries/CylinderGeometry.js",
   "three/src/geometries/PlaneGeometry.js",
   "three/src/geometries/SphereGeometry.js",
   "three/src/lights/AmbientLight.js",
   "three/src/lights/DirectionalLight.js",
   "three/src/lights/PointLight.js",
   "three/src/loaders/CubeTextureLoader.js",
   "three/src/loaders/TextureLoader.js",
   "three/src/materials/MeshBasicMaterial.js",
   "three/src/materials/MeshLambertMaterial.js",
   "three/src/materials/MeshPhongMaterial.js",
   "three/src/materials/MeshStandardMaterial.js",
   "three/src/materials/RawShaderMaterial.js",
   "three/src/materials/ShaderMaterial.js",
   "three/src/math/Box3.js",
   "three/src/math/Color.js",
   "three/src/math/Euler.js",
   "three/src/math/Matrix4.js",
   "three/src/math/Quaternion.js",
   "three/src/math/Sphere.js",
   "three/src/math/Vector2.js",
   "three/src/math/Vector3.js",
   "three/src/objects/Group.js",
   "three/src/objects/InstancedMesh.js",
   "three/src/objects/Mesh.js",
   "three/src/objects/Points.js",
   "three/src/renderers/WebGLRenderer.js",
   "three/src/scenes/FogExp2.js",
   "three/src/scenes/Scene.js"
  ],
  "milkis": [
   "node-fetch"
  ],
  "node-http2": [
   "node:http2"
  ],
  "webextension-polyfill": [
   "webextension-polyfill"
  ],
  "i18next": [
   "i18next",
   "i18next-browser-languagedetector"
  ],
  "supabase": [
   "@supabase/ssr",
   "@supabase/supabase-js"
  ],
  "lumi-components": [
   "jss",
   "jss-preset-default",
   "qrcode.react",
   "react-media-hook",
   "react-password-strength-bar"
  ],
  "marked": [
   "marked"
  ],
  "elmish": [
   "react-dom/server.js"
  ],
  "prospero": [
   "@fastify/multipart",
   "graphql",
   "node:crypto"
  ],
  "axon": [
   "bun",
   "node:http",
   "node:net",
   "node:stream",
   "stream"
  ],
  "markdown-it-js": [
   "markdown-it",
   "markdown-it-collapsible"
  ],
  "react-basic-dnd": [
   "react-dnd",
   "react-dnd-html5-backend",
   "react-dnd-test-backend",
   "react-dnd-touch-backend"
  ],
  "optparse": [
   "child_process"
  ],
  "node-path": [
   "node:path"
  ],
  "node-url": [
   "node:url"
  ],
  "z3": [
   "z3-solver"
  ],
  "ulid": [
   "ulid"
  ],
  "node-streams": [
   "node:stream"
  ],
  "leveldb": [
   "level"
  ],
  "whine-core": [
   "glob",
   "micromatch",
   "vscode",
   "vscode-languageclient/node.js",
   "vscode-languageserver-textdocument",
   "vscode-languageserver/node",
   "yaml"
  ],
  "ace": [
   "ace-builds",
   "ace/anchor",
   "ace/background_tokenizer",
   "ace/document",
   "ace/edit_session",
   "ace/editor",
   "ace/ext/language_tools",
   "ace/range",
   "ace/scrollbar",
   "ace/search",
   "ace/selection",
   "ace/token_iterator",
   "ace/tokenizer",
   "ace/undomanager",
   "ace/virtual_renderer"
  ],
  "node-stream-pipes": [
   "stream"
  ],
  "node-event-emitter": [
   "node:events"
  ],
  "golem-fetch": [
   "node:url"
  ],
  "express": [
   "cookie-parser",
   "express",
   "http",
   "https"
  ],
  "idb": [
   "idb"
  ],
  "yoga-elasticsearch": [
   "@elastic/elasticsearch"
  ],
  "mysql": [
   "mysql"
  ],
  "yoga-sqlite": [
   "@libsql/client"
  ],
  "meowclient": [
   "keys-converter",
   "meowclient"
  ],
  "n3": [
   "n3"
  ],
  "milkdown": [
   "@milkdown/crepe",
   "@milkdown/kit/core",
   "@milkdown/kit/utils"
  ],
  "yoga-acp-om": [
   "@anthropic-ai/claude-agent-sdk"
  ],
  "vitest": [
   "vitest"
  ],
  "ezfetch": [
   "buffer",
   "stream"
  ],
  "tanstack-query": [
   "@tanstack/react-query"
  ],
  "yoga-better-auth": [
   "better-auth",
   "better-auth/client",
   "better-auth/db",
   "better-auth/plugins",
   "pg"
  ],
  "node-execa": [
   "process"
  ],
  "webb-commandline": [
   "child_process"
  ],
  "csv-stream": [
   "csv-parse",
   "csv-stringify"
  ],
  "react-icons": [
   "react-icons/ai",
   "react-icons/bi",
   "react-icons/bs",
   "react-icons/cg",
   "react-icons/ci",
   "react-icons/di",
   "react-icons/fa",
   "react-icons/fa6",
   "react-icons/fc",
   "react-icons/fi",
   "react-icons/gi",
   "react-icons/go",
   "react-icons/gr",
   "react-icons/hi",
   "react-icons/hi2",
   "react-icons/im",
   "react-icons/io",
   "react-icons/io5",
   "react-icons/lia",
   "react-icons/lu",
   "react-icons/md",
   "react-icons/pi",
   "react-icons/ri",
   "react-icons/rx",
   "react-icons/si",
   "react-icons/sl",
   "react-icons/tb",
   "react-icons/tfi",
   "react-icons/ti",
   "react-icons/vsc",
   "react-icons/wi"
  ],
  "react-basic-emotion": [
   "@emotion/react"
  ],
  "faker-ffi": [
   "@faker-js/faker"
  ],
  "nextjs": [
   "next/document",
   "next/head",
   "next/image.js",
   "next/link",
   "next/navigation",
   "next/script",
   "swr"
  ],
  "node-workerbees": [
   "worker_threads"
  ],
  "jsdom": [
   "jsdom"
  ]
 },
 "broken_closure": [
  "ace",
  "address-rfc2821",
  "affjax-node",
  "axon",
  "benchlib",
  "blessed",
  "cbor-stream",
  "chartjs",
  "chartjs-halogen",
  "choku",
  "compile-fail",
  "css-class-name-extractor",
  "csv-stream",
  "debounce",
  "deno",
  "dotenv",
  "droplet",
  "echarts-simple",
  "elmish",
  "elmish-enzyme",
  "elmish-hooks",
  "elmish-html",
  "elmish-testing-library",
  "elmish-time-machine",
  "environment",
  "express",
  "ezfetch",
  "fahrtwind",
  "faker-ffi",
  "fakerjs",
  "framer-motion",
  "gojs",
  "golden-test",
  "golem-fetch",
  "graphql-client",
  "halogen-echarts-simple",
  "halogen-xterm",
  "httpurple",
  "hylograph-d3-kernel",
  "hylograph-simulation",
  "hylograph-simulation-halogen",
  "i18next",
  "idb",
  "ink",
  "jsdom",
  "leveldb",
  "logging-journald",
  "lumi-components",
  "markdown-it-js",
  "marked",
  "meowclient",
  "milkdown",
  "milkis",
  "motion",
  "mysql",
  "n3",
  "nanoid",
  "next-purs-rsc",
  "nextjs",
  "nextui",
  "node-buffer",
  "node-child-process",
  "node-event-emitter",
  "node-execa",
  "node-fs",
  "node-glob-basic",
  "node-http",
  "node-http2",
  "node-human-signals",
  "node-net",
  "node-os",
  "node-path",
  "node-process",
  "node-readline",
  "node-sqlite3",
  "node-stream-pipes",
  "node-streams",
  "node-tls",
  "node-url",
  "node-workerbees",
  "node-zlib",
  "oak",
  "oak-debug",
  "open-mkdirp-aff",
  "optparse",
  "org-doc",
  "postgresql",
  "prospero",
  "psa-utils",
  "react-aria",
  "react-basic-dnd",
  "react-basic-emotion",
  "react-basic-storybook",
  "react-dnd-kit",
  "react-icons",
  "react-markdown",
  "react-testing-library",
  "react-virtuoso",
  "reactix",
  "recharts",
  "rito",
  "rough-notation",
  "simple-ulid",
  "spec-discovery",
  "spec-node",
  "spec-reporter-xunit",
  "supabase",
  "systemd-journald",
  "tanstack-query",
  "toestand",
  "ts-bridge",
  "ulid",
  "url-regex-safe",
  "visx",
  "vitest",
  "webb-commandline",
  "webb-directory",
  "webb-file",
  "webb-test",
  "webextension-polyfill",
  "which",
  "whine-core",
  "xterm",
  "yaml-next",
  "yoga-acp-om",
  "yoga-better-auth",
  "yoga-config",
  "yoga-docker-compose",
  "yoga-dynamodb",
  "yoga-elasticsearch",
  "yoga-fastify",
  "yoga-fastify-om",
  "yoga-fetch",
  "yoga-fetch-om",
  "yoga-heroui",
  "yoga-jaeger",
  "yoga-next-fastify",
  "yoga-om-strom",
  "yoga-om-workerbees",
  "yoga-opentelemetry",
  "yoga-pino",
  "yoga-postgres",
  "yoga-react-native",
  "yoga-redis",
  "yoga-shadcn",
  "yoga-sqlite",
  "yoga-test-docker",
  "z3"
 ],
 "cut": [
  "affjax-node",
  "axon",
  "benchlib",
  "blessed",
  "cbor-stream",
  "compile-fail",
  "css-class-name-extractor",
  "css-frameworks",
  "csv-stream",
  "dotenv",
  "environment",
  "express",
  "ezfetch",
  "framer-motion",
  "golden-test",
  "golem-fetch",
  "graphql-client",
  "httpurple",
  "ink",
  "lumi-components",
  "marked",
  "meowclient",
  "motion",
  "next-purs-rsc",
  "node-buffer",
  "node-child-process",
  "node-event-emitter",
  "node-execa",
  "node-fs",
  "node-glob-basic",
  "node-http",
  "node-http2",
  "node-human-signals",
  "node-net",
  "node-os",
  "node-path",
  "node-process",
  "node-readline",
  "node-stream-pipes",
  "node-streams",
  "node-tls",
  "node-url",
  "node-workerbees",
  "node-zlib",
  "open-mkdirp-aff",
  "optparse",
  "org-doc",
  "pmock",
  "postgresql",
  "prospero",
  "psa-utils",
  "react-basic-dom-beta",
  "react-icons",
  "react-testing-library",
  "rito",
  "simple-ulid",
  "spec",
  "spec-discovery",
  "spec-mocha",
  "spec-node",
  "spec-reporter-xunit",
  "supabase",
  "test-unit",
  "ts-bridge",
  "webb-commandline",
  "webb-directory",
  "webb-file",
  "webb-test",
  "whine-core",
  "yoga-acp-om",
  "yoga-config",
  "yoga-docker-compose",
  "yoga-fastify-om",
  "yoga-fetch-om",
  "yoga-om-strom",
  "yoga-om-workerbees",
  "yoga-opentelemetry",
  "yoga-sqlite",
  "yoga-test-docker"
 ]
}
Scan script (run from staging/ after spago install)
import os, re, json, collections

SHIMMED = {"big-integer", "decimal.js", "react", "react-dom",
           "react-dom/client", "react-dom/server", "uuid"}
PAT = re.compile(
    r'''(?:\bfrom\s*|(?<!\w)import\s*\(\s*|(?<!\w)require\s*\(\s*)['"]([^'"\n]+)['"]''')
SKIP = {'node_modules', 'test', 'tests', '.git', 'examples', 'example', 'bench'}

bare = collections.defaultdict(set)
for pkgdir in sorted(os.listdir('.spago/p')):
    root = os.path.join('.spago/p', pkgdir)
    if not os.path.isdir(root): continue
    name = re.sub(r'-\d+\.\d+\.\d+.*$', '', pkgdir)
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = [d for d in dirnames if d not in SKIP]
        for f in filenames:
            if not f.endswith(('.js', '.mjs', '.cjs')): continue
            try: text = open(os.path.join(dirpath, f), encoding='utf-8', errors='replace').read()
            except OSError: continue
            for m in PAT.finditer(text):
                spec = m.group(1)
                if spec.startswith(('.', '/')) or ' ' in spec or spec.startswith('['): continue
                bare[name].add(spec)

unshimmed = {p: sorted(s - SHIMMED) for p, s in bare.items() if s - SHIMMED}

lock = json.load(open('spago.lock'))
deps = {p: set(v.get('dependencies', [])) for p, v in lock['packages'].items()}
rev = collections.defaultdict(set)
for p, ds in deps.items():
    for d in ds: rev[d].add(p)

broken = set(unshimmed) & set(deps)
frontier = list(broken)
while frontier:
    p = frontier.pop()
    for r in rev[p]:
        if r not in broken:
            broken.add(r); frontier.append(r)

print(json.dumps({"unshimmed": unshimmed, "cannot_run_closure": sorted(broken)}, indent=2))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate to Spago v1

2 participants